Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Subnet update error detection #991

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

yanjunz97
Copy link
Contributor

There are 2 issues in Subnet update error detection:

  • Operator will check the RealizedState right after we update the Subnet,
    while NSX may haven't updated the RealizedState for the updated Subnet.
    This PR add a delay before the RealizedState check for the Subnet update case.
  • Operator modifies the existing Subnet in store before updating the NSX Subnet,
    which results in the next reconcile will find the updated Subnet already in store,
    skip the update, and overwrite the Status as Ready.
    This PR uses a copy when updating the Subnet to avoid this issue.

@codecov-commenter
Copy link

codecov-commenter commented Jan 6, 2025

Codecov Report

Attention: Patch coverage is 78.94737% with 4 lines in your changes missing coverage. Please review.

Project coverage is 74.17%. Comparing base (57d792c) to head (ecf2948).
Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
pkg/nsx/services/subnet/subnet.go 78.94% 2 Missing and 2 partials ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #991      +/-   ##
==========================================
+ Coverage   73.43%   74.17%   +0.73%     
==========================================
  Files         118      118              
  Lines       16362    16358       -4     
==========================================
+ Hits        12015    12133     +118     
+ Misses       3564     3454     -110     
+ Partials      783      771      -12     
Flag Coverage Δ
unit-tests 74.17% <78.94%> (+0.73%) ⬆️
Files with missing lines Coverage Δ
pkg/nsx/services/subnet/subnet.go 69.88% <78.94%> (+3.98%) ⬆️

... and 4 files with indirect coverage changes

}
}
if !changed {
log.Info("Subnet not changed, skip updating", "SubnetId", uid)
return existingSubnet, nil
}
}
return service.createOrUpdateSubnet(obj, nsxSubnet, &vpcInfo)
return service.createOrUpdateSubnet(obj, nsxSubnet, &vpcInfo, isUpdate)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is a little confusing to have both changed and isUpdate, maybe just use changed & !existingSubnet to pass to the parameter in createOrUpdateSubnet

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated, thanks!

vpcSubnets[i].Tags = newTags
// Avoid updating vpcSubnets[i] to ensure Subnet store
// is only updated after the updating succeeds.
updatedSubnet := *vpcSubnets[i]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why we need two new variables comparableSubnet and updatedSubnet here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes sense to merge them, use only updatedSubnet instead, thanks!

@@ -398,7 +398,7 @@ func TestSubnetService_UpdateSubnetSet(t *testing.T) {
})

patchesCreateOrUpdateSubnet := gomonkey.ApplyFunc((*SubnetService).createOrUpdateSubnet,
func(r *SubnetService, obj client.Object, nsxSubnet *model.VpcSubnet, vpcInfo *common.VPCResourceInfo) (*model.VpcSubnet, error) {
func(r *SubnetService, obj client.Object, nsxSubnet *model.VpcSubnet, vpcInfo *common.VPCResourceInfo, isUpdate bool) (*model.VpcSubnet, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we need a new UT for createOrUpdateSubnet?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, some part of the code in createOrUpdateSubnet is not covered yet. Added the unit test, thanks

// For update case, wait for some time to make sure the realized state is updated.
if isUpdate {
time.Sleep(1 * time.Second)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's better that NSX fix that issue. After NSX patch API called, the realized status should be updated in sync mode instead

Copy link
Contributor Author

@yanjunz97 yanjunz97 Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened a bug for NSX team, let's see how they reply on this observations, thanks

@yanjunz97 yanjunz97 force-pushed the fix-subnet-update branch 2 times, most recently from 06fa16e to 534800e Compare January 7, 2025 11:43
Signed-off-by: Yanjun Zhou <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants